Using tablets to support self-regulated learning in a longitudinal integrated clerkship.

The need to train physicians committed to learning throughout their careers has prompted medical schools to encourage the development and practice of self-regulated learning by students. Longitudinal integrated clerkships (LICs) require students to exercise self-regulated learning skills. As mobile tools, tablets can potentially support self-regulation among LIC students.We provided 15 LIC students with tablet computers with access to the electronic health record (EHR), to track their patient cohort, and a multiplatform online notebook, to support documentation and retrieval of self-identified clinical learning issues. Students received a 1-hour workshop on the relevant features of the tablet and online notebook. Two focus groups with the students were used to evaluate the program, one early and one late in the year and were coded by two raters.Students used the tablet to support their self-regulated learning in ways that were unique to their learning styles and increased access to resources and utilization of down-time. Students who used the tablet to self-monitor and target learning demonstrated the utility of tablets as learning tools.LICs are environments rich in opportunity for self-regulated learning. Tablets can enhance students' ability to develop and employ self-regulatory skills in a clinical context

Fostering medical students' lifelong learning skills with a dashboard, coaching and learning planning.

IntroductionTo develop lifelong learning skills, students need feedback, access to performance data, and coaching. A new medical curriculum incorporated infrastructural supports based on self-regulated learning theory and the Master Adaptive Learner framework to engage students in reflection and learning planning. This study examines students' experience with a performance dashboard, longitudinal coaching, and structured time for goal-setting.MethodsFocus groups with first-year medical students explored performance dashboard usage, coaching and learning planning. We analyzed findings using thematic analysis. Results informed development of a 29-item survey rated strongly disagree (1) to strongly agree (5) to investigate experience with the dashboard, coaching and learning goals program. The survey was distributed to one first-year medical student class. We performed descriptive statistics and factor analysis.ResultsIn three focus groups with 21 participants, students endorsed using the dashboard to access performance information but had trouble interpreting and integrating information. They valued coaches as sources of advice but varied in their perceptions of the value of discussing learning planning. Of 152 students, 114 (75%) completed the survey. Exploratory factor analysis yielded 5 factors explaining 57% of the variance: learning goals development (α = 0.88; mean 3.25 (standard deviation 0.91)), dashboard usage (α = 0.82; 3.36 (0.64)), coaching (α = 0.71; 3.72 (0.64)), employment of learning strategies (α = 0.81; 3.67 (0.79)), and reflection (α = 0.63; 3.68 (0.64)).DiscussionThe student performance dashboard provides efficient feedback access, yet students' use of this information to guide learning is variable. These results can inform other programs seeking to foster lifelong learning skills

Performance of a cognitive load inventory during simulated handoffs: Evidence for validity.

BackgroundAdvancing patient safety during handoffs remains a public health priority. The application of cognitive load theory offers promise, but is currently limited by the inability to measure cognitive load types.ObjectiveTo develop and collect validity evidence for a revised self-report inventory that measures cognitive load types during a handoff.MethodsBased on prior published work, input from experts in cognitive load theory and handoffs, and a think-aloud exercise with residents, a revised Cognitive Load Inventory for Handoffs was developed. The Cognitive Load Inventory for Handoffs has items for intrinsic, extraneous, and germane load. Students who were second- and sixth-year students recruited from a Dutch medical school participated in four simulated handoffs (two simple and two complex cases). At the end of each handoff, study participants completed the Cognitive Load Inventory for Handoffs, Paas' Cognitive Load Scale, and one global rating item for intrinsic load, extraneous load, and germane load, respectively. Factor and correlational analyses were performed to collect evidence for validity.ResultsConfirmatory factor analysis yielded a single factor that combined intrinsic and germane loads. The extraneous load items performed poorly and were removed from the model. The score from the combined intrinsic and germane load items associated, as predicted by cognitive load theory, with a commonly used measure of overall cognitive load (Pearson's r = 0.83, p < 0.001), case complexity (beta = 0.74, p < 0.001), level of experience (beta = -0.96, p < 0.001), and handoff accuracy (r = -0.34, p < 0.001).ConclusionThese results offer encouragement that intrinsic load during handoffs may be measured via a self-report measure. Additional work is required to develop an adequate measure of extraneous load

Setting a common standard in clinical skills assessment: The experience of the California Consortium for the Assessment of Clinical Competence

Objective or purpose of innovation: To identify common clinical skills competency thresholds across schools by centralizing standard setting for a multi-institutional assessment. Background and/or theoretical framework and importance to the field: The California Consortium for the Assessment of Clinical Competence (CCACC) comprises 10 medical schools that administer a common multi-station clinical skills assessment (CPX). Previously, each institution determined their own, largely norm-referenced passing thresholds for the examination. With the elimination of USMLE Step 2 CS, there is a recognized need for robust clinical skills assessment beyond the individual institutional level. A collaboratively developed, multi-institutional examination with passing thresholds established via a rigorous process offers greater validity evidence for summative decisions made based on its results. Accordingly, the CCACC undertook centralized, criterion-based standard setting for the CPX. Design: Passing thresholds for the six core CPX cases were determined via two methods: modified Angoff, using expert raters from multiple institutions, and borderline regression, using global encounter ratings assigned by standardized patients. Results from the two methods were compared to each other and to institutions’ prior thresholds. Outcomes: Both methods yielded the same cumulative cut score based on averages across all cases (70%), but exhibited variation between individual cases, suggesting case-specificity. Compared with prior thresholds, some institutions’ pass rates would have been higher using the common criterion-referenced cut score, while others would have been lower. Innovation’s strengths and limitations: This study demonstrates the feasibility of centralizing standard setting across multiple institutions using two criterion-based methods. Standardized patient ratings may generate similar passing thresholds to those determined by clinicians. Further studies are necessary to determine whether these findings generalize to other case types and how best to apply centralized standards within each institution’s context. Feasibility and generalizability: The CCACC’s standard setting approaches may be applied across other institutions sharing an assessment, allowing for comparison of learner performance to a common standard. Given the similar results, the choice of method may be determined by resource availability

Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills

Self-efficacy is thought to be important for resuscitation proficiency in that it influences the development of and access to the associated medical knowledge, procedural skills and crisis resource management (CRM) skills. Since performance assessment of CRM skills is challenging, self-efficacy is often used as a measure of competence in this area. While self-efficacy may influence performance, the true relationship between self-efficacy and performance in this setting has not been delineated. We developed an instrument to measure pediatric residents’ self-efficacy in CRM skills and assessed its content validity, internal structure, and relationship to other variables. After administering the instrument to 125 pediatric residents, critical care fellows and faculty, we performed an exploratory factor analysis within a confirmatory factor analysis as well as a known group comparison. The analyses specified four factors that we defined as: situation awareness, team management, environment management, and decision making. Pediatric residents reported lower self-efficacy than fellows and faculty in each factor. We also examined the correlation between self-efficacy and performance scores for a subset of 30 residents who led video recorded simulated resuscitations and had their performances rated by three observers. We found a significant, positive correlation between residents’ self-efficacy in situation awareness and environment management and their overall performance of CRM skills. Our findings suggest that in a specific context, self-efficacy as a form of self-assessment may be informative with regards to performance

Assessment of Medical Students’ Shared Decision-Making in Standardized Patient Encounters

BackgroundShared decision-making, in which physicians and patients openly explore beliefs, exchange information, and reach explicit closure, may represent optimal physician-patient communication. There are currently no universally accepted methods to assess medical students' competence in shared decision-making.ObjectiveTo characterize medical students' shared decision-making with standardized patients (SPs) and determine if students' use of shared decision-making correlates with SP ratings of their communication.DesignRetrospective study of medical students' performance with four SPs.ParticipantsSixty fourth-year medical students.MeasurementsObjective blinded coding of shared decision-making quantified as decision moments (exploration/articulation of perspective, information sharing, explicit closure for a particular decision); SP scoring of communication skills using a validated checklist.ResultsOf 779 decision moments generated in 240 encounters, 312 (40%) met criteria for shared decision-making. All students engaged in shared decision-making in at least two of the four cases, although in two cases 5% and 12% of students engaged in no shared decision-making. The most commonly discussed decision moment topics were medications (n = 98, 31%), follow-up visits (71, 23%), and diagnostic testing (44, 14%). Correlations between the number of decision moments in a case and students' communication scores were low (rho = 0.07 to 0.37).ConclusionsAlthough all students engaged in some shared decision-making, particularly regarding medical interventions, there was no correlation between shared decision-making and overall communication competence rated by the SPs. These findings suggest that SP ratings of students' communication skill cannot be used to infer students' use of shared decision-making. Tools to determine students' skill in shared decision-making are needed